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(54) Method and system for creating a document having metadata 



(57) The present application describes systems and 
methods for creating a document having metadata us- 
ing a pointing device such as a digital pen. In one con- 



figuration, a digital pen captures pen stroke data and 
processes that data to determine metadata that the pen 
then writes to metadata storage located on the docu- 
ment. 
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Description 

[0001] The illustrative embodiments disclosed in the 
present application are useful in systems including 
those for creating documents having metadata and 
more particularly are useful in systems including those 
for creating documents having metadata using a point- 
ing device such as a digital pen. 
[0002] Traditional paper based forms are used in 
many situations. Typically, aform will include pre-printed 
information and pre-printed boxes for data input. Such 
a traditional form might be filled in be a salesman in the 
field. In such a situation, a salesman would fill in some 
information on the form and send the form to a central 
processing location by mail orfacsimile. The data on the 
form would typically be scanned or keyed in by an op- 
erator. 

[0003] A user may wish to attach metadata storage to 
a document such that the attached storage contains da- 
ta that is in some way related to the document that the 
storage device is attached to. The storage typically in- 
cludes an electronic version of the document in addition 
to the metadata that relates in some way to the informa- 
tion on the document. For example, a user may have an 
augmented document that is a piece of paper with an rf- 
id tag attached to it. The user may then write on the doc- 
ument with a pen, scan the document into a computer 
to create an e-copy of the written page. The user may 
then associate some metadata with the e-copy into the 
computer and then use a separate tag writer to write the 
tag. The computer repository then associates the e- 
copy with the physical document having the filled rf-id 
tag. Accordingly, a user must scan the document to cre- 
ate an e-copy of a document. 

[0004] A reference entitled Programmable Physical 
Document is shown in United States Patent Application 
Publication No. US 2002/008036 A1 , published June 
27, 2002 to Snowdon, et al. 

[0005] The present application describes illustrative 
embodiments of systems and methods for processing 
forms using a digital pointing device. In further illustra- 
tive embodiments, a user utilizes a digital pen to asso- 
ciate metadata with a physical document having meta- 
data storage. In a further illustrative embodiment, the 
digital pen captures stroke data for a document, obtains 
metadata for the document and facilitates storage of the 
metadata in the document metadata storage. In another 
further embodiment, the digital pen stroke data is used 
to determine metadata and the digital pen stores the 
metadata in the document metadata storage. 
[0006] FIG. 1 is a schematic representation of a digital 
pen system according to an illustrative embodiment of 
the present application. 

[0007] FIG. 2 is a schematic representation of a doc- 
ument having metadata storage according to an illustra- 
tive embodiment of the present application. 
[0008] FIG. 3A is a flow chart showing a process for 
identifying a form version according to the illustrative 



embodiment of the present application. 
[0009] FIG. 3B is a flow chart showing a process for 
updating a form version database according to the illus- 
trative embodiment of the present application. 

5 [0010] A method and system for generating docu- 
ments is shown. In an illustrative embodiment, a system 
and method for using a digital pen to input data on a 
physical media is shown. Metadata may be associated 
with the paper document and stored into a metadata 

10 storage such as an rf-id tag or an integrated circuit at- 
tached to the media. 

[0011] A type of physical document includes media 
such as paper that is used for writing. Metadata that is 
associated with a physical document may include an 

15 identification code and data that is associated with the 
physical document and may be relevant to the writing 
on the document. An augmented document is a physical 
document such as a piece of paper or spiral bound book 
that includes storage for metadata. The storage may in- 

20 elude an rf-id tag, an integrated circuit or other storage 
device for storing metadata. An e-copy of a document 
is an electronic version of a physical document that may 
contain a rasterized or character based representation 
of a physical document and may include the metadata 

25 associated with the document. Here, an Information 
Document is an augmented document that includes an 
e-copy having metadata and that is reflected in a data- 
base as associating the augmented document and the 
e-copy. 

30 [0012] Digital pointing devices such as the pen and 
tablet systems available from Wacom Technology Corp. 
of Vancouver, Washington, USA, allow a user to utilize 
a computer to capture pen strokes that the user traces 
over a tablet. Digital pointing devices may utilize capac- 

35 jtive sensors or other grid based sensors to determine 
absolute or relative position of the pointing device. 
[0013] Digital pens allow a user to capture or digitize 
handwriting or pen strokes that the user writes on a me- 
dium such as a piece of paper by using a processor such 

40 as a personal computer. Certain digital pens utilize an 
imaging device to scan or record an image of the pen 
stroke. Certain other digital pens use mechanical sen- 
sors in order to record a pen stroke. The pen systems 
may utilize positioning systems such as light-based 

45 scanning systems including infrared (ir) sources and de- 
tectors in order to determine an absolute or relative po- 
sition of the pen. Digital pen systems include the 
N-Scribe system available from Digital Ink of Wellesley, 
MA, USA, and the E-Pen system available from E-Pen 

50 inMotion of Matam, Haifa Israel. A digital pointing device 
includes the V-Pen system available from OTM Tech- 
nologies of Herzliya Israel. 

[0014] Another digital pen system is the Sony-Erics- 
son CHA-30 Chatpen and Anoto paper available from 
55 Anoto AB of Sweden. The Chatpen utilizes a Bluetooth 
transceiver in order to communicate with a processor 
and the Anoto paper includes a grid encoding informa- 
tion such position information that is detected by the 
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Chatpen. Additional information may be captured in- 
cluding information related to pressure, speed and pen 
attitude. The additional information includes biometric 
information that may be used to identify or authenticate 
a user. 

[0015] A reference describing a handheld writing de- 
vice is shown in U.S. patent number 6,050,490, issued 
April 18, 2000. A reference entitled method and appa- 
ratus for wireless remote information retrieval and pen- 
based data entry is shown in United States Patent 
Number 5,561 ,446 issued October 1 , 1 996 to Montlick. 

The Digital Pen 

[0016] The present application describes systems 
and methods forcomposing documents using a pointing 
device. The illustrative embodiments described herein 
utilize a Chatpen digital pen in order to compose a doc- 
ument and indicate delivery information for a completed 
document. However, other sensing devices may be uti- 
lized to compose other documents. 
[001 7] The digital pen is utilized to capture information 
regarding the pen strokes made by a user. The proces- 
sor includes a micro-controller such as an 8051 -based 
micro-controller with code to perform the metadata and 
communications routines. 

[0018] Toolkits for the development of the processes 
and stroke data interpretation are known as is process- 
ing biometric data including stroke data. 
[0019] However, other digital pens may also be uti- 
lized. Certain digital pens utilize position determination 
be determining the actual location of the pen on a piece 
of paper in order to provide a relative location in terms 
of the location in the space of the piece of paper. Certain 
digital pens scan the ink as it is applied to digitize a 
stroke while others sense the stroke using sensors such 
as pressure sensors, Doppler sensors, accelerometers 
and other sensing mechanisms. 
[0020] The illustrative embodiments describe meth- 
ods and apparatus for capturing pen strokes and for 
capturing form identification information. The form iden- 
tification data may include a form serial number, form 
number and version number. The form identification in- 
formation may be captured using digital pen stroke data. 
Additional methods and apparatus for sensing the form 
version are described. The processes and apparatus 
described may be implemented using hardware, soft- 
ware or a combination of both. The communications 
channels may be wireless or wired and may utilize se- 
curity techniques such as encryption. The data storage 
and data processors may be locally or remotely located. 

The Document Composition System 

[0021] Referring to FIG. 1 , a first illustrative embodi- 
ment describing a digital pen form verification system is 
shown. 

[0022] Digital Pen 100 includes a processor 114, 



memory 112, ink 117, a camera or image sensor 115, a 
battery 1 1 6 and a wireless transceiver 111. The pen 1 00 
includes an LED 119. It also includes pen stroke data 
and biometric sensors (not shown). In an alternative, the 
5 ink 1 1 7 is machine detectable. In another embodiment, 
the ink is invisible. In an alternative, the pen includes 
other visual indicators such as an LCD display. In an 
alternative, the pen includes audio indicators such as a 
speaker, buzzer or speech synthesizer. In another em- 
bodiment other feedback devices such as tactile feed- 
back are provided. 

[0023] The pen 1 00 includes a pen tip (not shown) that 
writes using the ink 117. Writing sensors (not shown) 
provide data regarding the stroke such as pressure, 
speed and pen attitude. The pen includes an ink-inhib- 
iting device so that the processor 1 1 4 can stop the flow 
of ink 117. The memory 114 includes a non-volatile 
memory. The pen 100 includes a real time clock (not 
shown). Alternatively, a counter may be utilized. The 
pen 1 00 utilizes a wireless transceiver 1 1 1 that is a Blue- 
tooth TM transceiver. Alternatively, other wireless com- 
munication channels can be utilized. In another alterna- 
tive, a wired communications channel such as a docking 
station may be utilized in addition to or as a replacement 
for the wireless transceiver. 

[0024] In another alternative, the pen includes audio 
input/output including synthesized voice output and 
voice recognition. Visual output is provided using an 
LCD display and LEDs and tactile feedback is provided 
using servomechanisms. Physical input includes an in- 
put button. 

[0025] Bluetooth TM utilizes several layers of security. 
At a link level, remote/local device authentication is re- 
quired before any communication can take place. At the 
Channel level, a link level connection occurs and then 
the devices need to authenticate before a communica- 
tions channel is established. Additionally, the data pay- 
load being transmitted may be encrypted. In this embod- 
iment, appropriate security at several protocol layers is 
utilized including the application layer. 
[0026] The embodiments described herein may utilize 
biometric data for purposes including identification and 
authentication of a user. In another embodiment, the 
digital pen is a Chatpen digital pen available from Anoto, 
AB. The pen provides biometric data relating to the pen 
strokes used including hand speed, pen tip pressure 
and the inclination angle between pen and paper. Such 
data is referred to herein as BIODATA. In alternative em- 
bodiments, the BIODATA may include other biometric 
data such as a retinal scan or fingerprint scan performed 
using an external processor such as processor 1 50. The 
pen is assigned a unique identification code that is a 
unique serial number for the pen. In an alternative, the 
PUID is a Bluetooth TM MAC code or other unique or 
group assigned code. 

[0027] The system includes at least one pen 1 00 that 
establishes a personal area network using Bluetooth 
TM. The paired device may be a Bluetooth TM router or 
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other processor 1 50 that connects to the digital pen 1 00 
using wireless connection 120 and provides a gateway 
using communications connection 134 to the Internet 
130. The paired device may include a cellular telephone 
or PDA 160 that has a Bluetooth connection 122 and a 
connection 136 to the Internet 130. 
[0028] Here, the system includes a server 1 40 that in- 
cludes storage 146 connected by connection 144 to 
processor 142. The server 140 is connected to the In- 
ternet 130 using communications channel 132. The 
server hosts a form version database that is periodically 
downloaded to the pen 1 00. Alternatively, processor 1 50 
or cellular device 160 includes the form version data- 
base (not shown) that can be uploaded to the pen 1 00. 
Server 140 or processor 150 may be utilized for other 
digital pen back end activities including pen stroke data 
analysis and background pattern lookup services. 
[0029] Metadata storage device 1 70 is an rf-id tag that 
is connected to the digital pen 1 00 using a wireless com- 
munications channel such as Bluetooth. The pen 110 
can read and write data to the metadata storage device 
170. 

[0030] It is contemplated that all of the connections 
utilize appropriate security measures. Other well-known 
input devices, servers, processors, networks and com- 
munications mechanisms may be used. A back-end ap- 
plication may be utilized to process pen strokes. The 
back end application then recognizes command strokes 
or strokes in command locations identified by the pat- 
tern. The data written by a user in a particular data input 
flied can be rasterized and then subjected to Optical 
character recognition (OCR) in orderto identify the data 
written by the user. 

[0031] Processor 150 utilizes a mobile Pentium 4 
processor and Windows XP. The server processors are 
geographically and load balanced application servers 
using systems availablefrom Sun Microsystems and the 
storage servers use multiple location redundant backup 
systems. Additionally, other appropriate wireless and 
wired networks and connections may be utilized. It is 
contemplated that other communications channels such 
as OC-3 lines or wireless connections could be used in 
place of the T1 lines. Similarly, the other communica- 
tions channels could be replaced with alternatives. As 
can be appreciated, various communication flows may 
be utilized, some of which will be chattier than others. 
[0032] The present embodiment may alternatively 
use any pen or stylus like device that provides for elec- 
tronically recording strokes. Position information may be 
processed into strokes ortransmitted in a separate data 
stream. 

[0033] The digital pen 1 00 approximates the size of a 
traditional pen and may be used by a user to handwrite 
information. 

[0034] Referring to FIG. 2, a schematic representa- 
tion of an augmented document is shown. Document 
200 includes an Anoto pattern 202 and includes a meta- 
data input section 204. There is a start process box 21 8 



and a metadata storage device such as an rf-id tag 270. 
In an alternative, there is no separate metadata input 
section or process command box. Additionally, the 
metadata storage device can alternatively be an inte- 
5 grated circuit or other storage device. 

[0035] Referring to FIG. 3A, flowchart describes the 
process of producing an Information Document (Info- 
Doc). The process 300 starts in step 305 and proceeds 
to initialization in step 31 0 in which the process receives 
10 pen strokes. In step 315, the process polls for a meta- 
data mode such as metadata entry strokes. The process 
proceeds to step 320 to determine if a process com- 
mand has been entered using process box 218. If so, 
the metadata entered in step 315 is sent to an external 
15 processor 150 in step 325. In step 330 the process re- 
ceives a metadata message from processor 150 and 
programs the metadata message into metadata storage 
device 270. The process then ends in step 340. The 
metadata may be an e-copy including an electronic copy 
20 of the document and the metadata information. 

[0036] In an alternative, the metadata message is pro- 
duced in the pen. The processor may be resident in the 
pen 100 or in an external processor 150, 142 or 160. 
The Chatpen and Anoto pattern provide location infor- 
ms mation, but in an alternative, other digital pen devices 
may be utilized. 

[0037] In order to create an InfoDoc, the user starts 
with a blank augmented document. As the user writes 
onto the augmented document 200, the pen captures 

30 what is written on the document in area 202. After the 
user is done writing the document, the user fills in some 
metadata fields regarding the document on an area 204 
of the document 200 allocated for metadata. The pen 
also captures the written metadata. When the user is 

35 ready to process the augmented data, the user checks 
the Process box 218. The pen then sends the written 
data and metadata to a processor 1 50 using a wireless 
communications channel 120. The written data is then 
either converted to text or imaged. This converted data 

40 is considered the electronic document portion of the e- 
copy. The metadata is converted to text and assigned a 
unique id such as a serial number. The processor 150 
sends the e-copy with metadata to a file repository 1 40 . 
The computer also sends the metadata text back to the 

45 pen. The pen 1 00 includes an rf-id tag writer and when 
it receives the metadata, it writes the metadata to the 
tag on the augmented document. 
[0038] In an alternative, the metadata and electronic 
version of the document is saved in the tag as an e-copy. 

so [0039] Referring to FIG. 3B, a process 350 for deter- 
mining and storing metadata is shown. The process 350 
starts in 355 and proceeds to capture pen data in meta- 
data mode step 360. In step 365 the pen digitizes the 
pen strokes and creates an e-copy of the document in 

55 the pen. In step 366, the pen then senses that the doc- 
ument is done, by the use of a signature block, a check 
box, a duration of time or other method and calculates 
metadata based upon the pen strokes such as the bio- 
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metric signature of the pen strokes. In step 367 the pen 
1 00 associates the metadata into the e-copy and in step 
368, the pen writes the metadata to the tag. The process 
ends in step 370. 

[0040] In another alternative, the process continually 
updates the metadata and does not sense an end of 
document indicator. In another alternative, the pen proc- 
esses input strokes in order to determine if it has enough 
data to create the biometric signature. The pen then 
writes the metadata without the e-copy of the document. 
[0041] Known systems such as C++ or Word and VBA 
may be utilized. The Anoto toolkit may be utilized. The 
authentication data may be used to ensure that only au- 
thorized users have access to the forms and data. In an 
alternative applicable to any of the embodiments de- 
scribed, the form processor can use the pen stroke data 
and pen identification data to determine if the salesman 
or user has authority to use the particular form, to deal 
with the particular client, or to sell the particular items 
requested. If the user is not authorized, the ink supply 
1 1 7 could be disabled or the pen otherwise disabled. 
[0042] In another embodiment, the form version indi- 
cator is a traceable indicator such as an alphanumeric 
sequence of characters that the user traces to inputform 
version data. In any of the described embodiments, er- 
ror indications or messages may be communicated us- 
ing LED indicators, a display such as an LCD display or 
audio indications such as beeps or synthesized speech 
using a speaker. Additionally, tactile feedback may be 
utilized to indicate an error condition or provide other 
feedback such as an indication that a form was success- 
fully sent. 

[0043] In another alternative applicable to any of the 
embodiments, Wi-Fi enabled wireless systems are uti- 
lized and the external processor comprises a Wi-Fi ca- 
pable hand-held pocket PC such as the Toshiba e740 
Pocket PC. Furthermore, differing types of processors 
and logic systems may be supported. For example, 
JAVA based PALM OS devices may be utilized. The 
message logic, processing logic, security logic, user in- 
terface logic, communications logic and other logic 
could be provided in JAVA format or in a format compat- 
ible with individual platforms such as Windows CE and 
PALM OS. Similarly, other portable computing devices 
such as laptop computers and tablet computers and 
wireless capable computers could be utilized. Other 
platforms such as those using Symbian OS or OS-9 
based portable processors could be utilized. 
[0044] The present application describes illustrative 
embodiments of a system and method for composing 
documents using a pointing device and in one embodi- 
ment describes a digital pen system for composing and 
sending a completed form to a destination indicated by 
the user using the digital pen. The embodiments are il- 
lustrative and not intended to present an exhaustive list 
of possible configurations. Where alternative elements 
are described, they are understood to fully describe al- 
ternative embodiments without repeating common ele- 



ments whether or not expressly stated to so relate. Sim- 
ilarly, alternatives described for elements used in more 
than one embodiment are understood to describe alter- 
native embodiments for each of the described embodi- 

5 ments having that element. 

[0045] The described embodiments are illustrative 
and the above description may indicate to those skilled 
in the art additional ways in which the principles of this 
invention may be used without departing from the spirit 

10 of the invention. Accordingly, the scope of each of the 
claims is notto be limited by the particular embodiments 
described. 



15 Claims 

1 . A method for associating metadata with a document 
having a metadata storage device comprising: 

20 initializing a pointing instrument; 

recognizing a metadata mode; 

sending metadata data to a processor; 

receiving metadata from the processor; and 

storing the metadata in the metadata storage 
25 device. 



2. The method of claim 1 further comprising: 

receiving a process metadata command, 
wherein the pointing instrument is a digital 



30 



pen. 



3. The method of claim 2 wherein the metadata data 
35 is pen stroke data captured from a predefined area 

of the document. 

4. The method of claim 2 wherein the metadata stor- 
age device is an rf-id tag. 

40 

5. The method of claim 2 wherein the metadata data 
is pen stroke data captured from all pen strokes 
made on the document. 

45 6. The method of claim 2 wherein the metadata data 
is pen stroke data captured from a first subset of all 
pen strokes made on the document. 

7. The method of claim 2 wherein the metadata in- 
50 eludes biometric data. 

8. The method of claim 2 further comprising: 



55 



storing an e-copy of the document strokes to 
the metadata storage device. 
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